Optimizing drive schemes for multiple projector systems

ABSTRACT

Light projection systems and methods may comprise combining light from two or more projectors. Each projector may be controlled so that the combined light output of the projectors matches a target for the projected light. In some embodiments optimization is performed to generate image data and control signals for each of the projectors. Embodiments may be applied in image projecting applications, lighting applications, and 3D stereoscopic imaging.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. application Ser. No.15/312,165. U.S. application Ser. No. 15/312,165 is the US NationalStage of PCT Application No. PCT/CA2015/000324 filed 15 May 2015, whichclaims priority from U.S. application Ser. No. 61/994,002 filed 15 May2014 and U.S. Patent Application No. 62/148,041 filed 15 Apr. 2015. Forpurposes of the United States, this application claims the benefit under35 U.S.C. § 119 of U.S. Application No. 61/994,002 filed 15 May 2014entitled BRIGHTNESS BOOSTER FOR MULTIPLE-STAGE PROJECTORS and U.S.Patent Application No. 62/148,041 filed 15 Apr. 2015 entitled OPTIMIZINGDRIVE SCHEMES FOR MULTIPLE PROJECTOR SYSTEMS, both which are herebyincorporated herein by reference for all purposes.

FIELD

This invention relates to image projectors and methods for projectingimages. The invention has application, for example, in cinemaprojection, projection television, advertising displays, generalillumination such as spatially adaptive automotive headlights and thelike.

BACKGROUND

Many light projectors have a light source that uniformly illuminates animage formation chip, such as a DMD, LCoS, LCD or reflective LCD (orfilm) that subtractively modulates the incoming light in order to createa target image. Such projectors typically 1) cannot exceed a peakluminance set by the optical power of the light source, the projectedimage size, and the reflectivity of the image screen, and 2) have adynamic range or contrast that is limited by the image formation device,for example film, or digital devices like LCD, LCOs or DMD imagingchips.

Light projectors vary in their capability to produce target images withspecified luminance and chromaticity values. The range of capabilitiesstem from technological limitations related to maximum peak luminance(optical output of the light source) to lowest black-level and hencecontrast (contrast of the included image formation technology), tochromatic purity and colour gamut (governed either by the filtersapplied to a broadband source or to the wavelength of, for example, alaser light source), as well as uniformity and noise specifications.Some projectors can produce light output with limited contrast, forexample reaching a peak luminance of 100 cd/m² and a black level of 1cd/m², and hence a contrast of 100:1. Other projectors can reachbrighter highlights (by increasing the light source power), and/ordeeper black levels (using higher contrast image formation technology).In some systems, very deep black levels can be achieved by modulatingthe image twice (“dual modulation”). The contrast or dynamic range of aprojector can be dynamically adjusted by inserting an iris or aperturein the light path, whose light blocking may be driven in response toimage content.

The type of and requirements of image or video content to be reproducedon a projector can vary significantly in time over the course of apresentation of image or video content. The presentation could, forexample, comprise presentation of a movie in a cinema, a liveperformance that uses projectors, or projection of light by adaptive(image-) projector headlights while driving in different conditions in avehicle. For example a movie might begin with a dark, high contrast,black and white scene, and later contain bright and low contrast sceneswith pure colors. While driving at night, an adaptive car headlightmight be required to project a uniform, and low contrast light field onan empty road outside the city, but within the city be required toproduce a very high contrast, bright image to highlight stop signs,avoid illuminating upcoming cars (casting a shadow in that region) orsignaling information on the road.

High brightness, high dynamic range projectors are often more expensivethan standard lower dynamic range projectors for similar average light(power) outputs. One reason for this is that achieving better blacklevels often requires more elements within the system (for example dualmodulation designs that use cascaded, light attenuating elements).Another reason is that achieving higher peak luminance on the samescreen requires more light-source power in the projector.

There remains a need for good ways to control a projection system toreproduce image content having characteristics that vary significantlyover time (e.g. characteristics such as dynamic range, black level,maximum luminance, color saturation) as in the examples above. Such wayswould beneficially provide advantages such as reducing powerrequirements, providing good black level, and/or providing brighthighlights.

There remains a need for light projection systems that offer one or bothof higher image quality and better cost efficiency.

There remains a need for practical and cost effective projection systemssuitable for projecting patterns such as images, desired lampillumination patterns, and the like. There is a particular need for suchsystems that are able to faithfully display content havingcharacteristics that change significantly over time (e.g. systems calledupon to display bright low-contrast images at some times and to displaydark images with bright highlights at other times).

SUMMARY

This invention has a number of aspects. One aspect provides a projectorsystem that combines a plurality of projectors. The projectors may haveperformance characteristics different from one another. The projectorsmay be separate devices or share certain components, such as controlelectronic or certain optical elements. Another aspect provides controlhardware devices useful for coordinating the operation of two or moreprojectors to display an image. Another aspect provides a method forsplitting an incoming image signal into separate images.

Multiple image generating devices may be used to form a combined image.Each device has a set of operating specifications (which may include,for example, specifications such as peak luminance, resolution, blacklevel, contrast, chromatic extent or gamut). Defined mathematicalfunctions provide image quality and cost metrics in a mathematicalframework that permits optimization to achieve goals such as improvedimage quality or lower cost. The results of the optimization yieldseparate image data for each image generating device.

This concept can be applied to projectors, where two or more systemswith similar or different capabilities produce a combined image inaccordance with image data.

In cases where a low dynamic range projector is present in aninstallation or a high dynamic range projector of suitable maximumoutput power cannot be found, it may be desirable to combine two or moreprojectors with similar or different capabilities in order to create asingle image with high peak luminance and low black levels. An exampleof such an arrangement comprises a low dynamic range projector and ahigh dynamic range projector to create a single image with high peakluminance and low black levels.

Further aspects and example embodiments are illustrated in theaccompanying drawings and/or described in the following description.

BRIEF DESCRIPTION OF THE DRAWINGS

The accompanying drawings illustrate non-limiting example embodiments ofthe invention.

FIG. 1 is a block diagram showing a projection system according to anexample embodiment.

FIG. 2A is an example image. FIG. 2B and FIG. 2C are respectively imagesprojected by an LDR projector and an HDR projector that may be combinedto reproduce the image of FIG. 2A.

FIG. 3A is another example image. FIG. 3B and FIG. 3C are respectivelyimages projected by an LDR projector and an HDR projector that may becombined to reproduce the image of FIG. 3A.

FIG. 4A is another example image. FIG. 4B and FIG. 4C are respectivelyimages projected by an LDR projector and an HDR projector that may becombined to reproduce the image of FIG. 4A.

FIG. 5A is another example image. FIG. 5B and FIG. 5C are respectivelyimages projected by an LDR projector and an HDR projector that may becombined to reproduce the image of FIG. 5A.

FIG. 6A is another example image. FIG. 6B and FIG. 6C are respectivelyimages projected by an LDR projector and an HDR projector that may becombined to reproduce the image of FIG. 6A.

FIG. 7 is a schematic illustration of an abstract conception of adisplay.

FIG. 8 illustrates two displays acting serially.

FIG. 9 illustrates two displays acting in parallel.

FIG. 10 is a block diagram illustrating an example compound display.

FIG. 11 is a block diagram illustrating a system in which displayparameter optimization is performed to determine the parameters andillumination to be used to reproduce an input target image using adisplay.

FIG. 12 is a flowchart illustrating the combination of images from firstand second projectors to yield an output image.

FIG. 13 is a flowchart illustrating a method for determining what imagewill be shown by each of a plurality of projectors to yield a targetimage.

FIG. 14 is block diagram illustrating a projection system with aindependent main and auxiliary light source (“boost light source”) aswell as two imaging elements that can steer or attenuate light onto ascreen.

FIG. 15 is a flow chart illustrating how to control the light sources ofa projection system with a main and an auxiliary (boost) light source.

FIG. 16 illustrates example image data with different imagecharacteristics and the corresponding intensity settings (controlsignals) for an auxiliary (boost) light source.

DETAILED DESCRIPTION

Throughout the following description, specific details are set forth inorder to provide a more thorough understanding of the invention.However, the invention may be practiced without these particulars. Inother instances, well known elements have not been shown or described indetail to avoid unnecessarily obscuring the invention. Accordingly, thespecification and drawings are to be regarded in an illustrative, ratherthan a restrictive sense.

One motivation for combining two or more low dynamic range projectors(projector tiling), or even two low peak luminance, high contrast(dynamic range) projectors, is to boost the overall luminance(brightness) on screen of the resulting image. Low dynamic rangeprojectors are common and a commodity technology and thus command a muchlower purchase price than high dynamic range projectors of similar totaloutput brightness.

FIG. 1 schematically illustrates a projector system comprising aplurality of projectors.

In some embodiments, all of the plurality of projectors contribute lightto the same viewing area (e.g. boundaries of the fields of view of theprojectors may be the same). Each of the plurality of projectors maydeliver light to any part of the viewing area. Viewers perceive thecombined output of the projectors. In some embodiments, each of theprojectors projects onto the full display area of the viewing screen.

In a system where a low and high dynamic range projector (LDR and HDR)are combined, the optimal ratio of light contributed by each of theprojectors to the final image can vary greatly. This variation is aresult of image and environmental properties such as:

-   -   ambient light level at the screen location    -   image peak luminance    -   image average luminance    -   light output of both projectors    -   efficiency of both projectors (lumens/watt)    -   minimum black level of LDR projector    -   amount of black in the image    -   proximity of black to bright features (veiling luminance)    -   the presence of non-uniformity (or other artifacts) in the LDR        projector that can be corrected by the HDR projector    -   the presence of speckle (or other artifacts) in the HDR        projector that can be reduced through use of the LDR projector    -   ability to reduce power consumption of the projectors by showing        dimmer content (for power consumption optimization)

Below are five example cases showing how images from a HDR projector anda LDR projector can be combined according to an example embodiment ofthe invention. “Bright” and “dim” refer to the luminance level of theimage.

Case 1: Bright Low Dynamic Range Image, Elevated Black Levels

The image (FIG. 2A) has a high black level. Darker details aresurrounded closely by white features. In this example case the desiredbrightness of the image exceeds the capability of the LDR projector.

The LDR projector may be controlled to output as much light as it can(see FIG. 2B) and the HDR projector may be controlled to supplement someof the brighter features to simply increase the overall brightness ofthe image as shown in FIG. 2C.

Case 2: Dim Low Dynamic Range Image, High Blacks

This image (FIG. 3A) does not have very high dynamic range. The LDRprojector is sufficiently bright to produce the image at the desiredlevel. In this case the LDR projector may simply show the input image“as is” (FIG. 3B) and the HDR projector may output nothing or be off(FIG. 3C).

Case 3: Bright High Dynamic Range Image, High Blacks

This image (FIG. 4A) shows some detail in the darker areas so the imagedoes not have a very low black level. Brighter parts of the image exceedthe brightness capability of the LDR projector. The LDR projector maydisplay an image as shown in FIG. 4B and the HDR projector may displayan image as shown in FIG. 4C.

Case 4: Bright High Dynamic Range Image, Low Blacks

This image (FIG. 5A) has very low back levels with complete absence ofdetail in the darks. Due to the high expected brightness of the candleflame, the LDR projector may be turned off altogether, or dimmed down bythe use of an iris (FIG. 5B), and the HDR projector may produce theentire image (FIG. 5C).

Case 5: Dim Low Dynamic Range Image, Low Blacks

Here the peak brightness of the image is quite low (see FIG. 6A) and atthe same time the black levels are also very low. The LDR projectorwould need an Iris over the lens (detailed below) to get the blacklevels down sufficiently. In this case the peak brightness through thepartially closed Iris would be sufficient to display the image so theHDR projector would not be needed. FIG. 6B shows the image output by theLDR projector with an iris partially closed. FIG. 6C shows the(black/null) output of the HDR projector.

Iris/Global Lamp Power Control

Low dynamic range projectors often produce a dark grey image whenattempting to show black due to limitations of light-modulatortechnology. As an example, consider images in which the brightest areashave luminances lower than the peak luminance of the projector. Here,better contrast can be achieved by dimming the light source. In anotherexample, the amount of detail in dark areas of a target image can bedetermined to be of higher perceptual importance to the viewer. In suchcases, bright content may be sacrificed by dimming the projector toregain deeper black levels. Most low dynamic range projectors are lampbased and cannot easily be dimmed or turned on and off (to create pureblack) on a per scene basis due to warm-up issues.

In cases where a low dynamic range projector needs to be turned “off” orsimply down, an iris can be placed in the optical path (e.g. over thelens). The iris may then be made smaller to improve the black level ofthe projected image. Also note that the iris is not binary; an iris maybe opened to a size dictated by the desired image black level. It isassumed that the iris can change size with sufficient speed as to notcreate a noticeable lag when changing scenes. The iris function may alsobe implemented by some other electrical or mechanical means such as anLCD plate (electrically dimmable) or a high speed shutter rapidlyclosing and opening.

If the LDR projector has a solid state light source that has a lightoutput that can be controlled, an iris may not be needed. In suchembodiments, the light source may be dimmed in an amount such that itslight output is equivalent to the light that would have been availablethrough a constricted iris.

A high dynamic range projector may optionally include a globallydimmable solid state light source and/or an iris.

Artifact Mitigation

It may be advantageous for image quality to never completely close theiris and accept a slightly higher black level. If a HDR projector showspoorer image quality due to field non-uniformity or other artifacts,having at least a base amount of light from the LDR projector can helpto perceptually mitigate the artifacts.

If an LDR projector displays image artifacts such as vignetting or othernon-uniformity, the HDR projector may be used to correct for thenon-uniformity of the light field.

Projector Balancing Algorithm

Display Representation

In order to determine settings for each component projector one can takethe capabilities of each projector into account.

Previous approaches commonly model image formation as a simple pipelinewhere each component takes an input, operates upon it, and passes it tothe next stage. This approach is effective for systems consisting ofrelatively few controllable elements, e.g. light sources, modulators oririses, coupled with relatively many passive optical components such asmirrors or lenses, however it is less desirable in more complex systems.Such systems may combine multiple displays (projectors) or feed theoutput of one display into subsequent displays. In this case, parametersfor later stages of the pipeline can be adjusted in order to compensatefor artifacts or performance limitations of earlier stages.

It is advantageous to think of each display in an abstract sense astaking a set of display parameters, P (e.g. pixel values), and a sourceillumination, S, which are then operated upon by the display to producean output image, O=F(P,S), where the function F models the operation ofthe specific display hardware. This abstract conception of a display isillustrated in FIG. 7.

This modular approach allows displays to be nearly arbitrarily connectedto form networks of abstract displays and passive optical components tomodel more complex imaging systems. Displays in a network can beconnected either in series to form a single optical path, or in parallelto combine multiple optical paths, or in a combination of serial andparallel designs.

An example of a serial connection for two displays is shown in FIG. 8for a system comprising two amplitude modulators connected in series.Such an arrangement is used in some Extended Dynamic Range (EDR)projectors which compensate for limited contrast ratios of individualamplitude modulators by cascading the modulators. The output contrast isconsequently the product of the contrast ratios of the two modulators.

An example of a parallel arrangement is found in projectorsuper-resolution applications, in which the output images from multipleprojectors are overlapped with a slight deregistration in order togenerate higher spatial frequency features than are present in an imagefrom a single projector. This arrangement is shown in FIG. 9.

In the parallel arrangement, the optical paths of two amplitudemodulating projectors are combined (by the projection screen) to producean output image.

Based on the arrangement, the output image can be determinedmathematically by either addition or composition of images generated bythe component displays. Taking two displays with functions F₁ and F₂taking parameters P₁ and P₂ respectively, a parallel configurationresults in the following expression for the output image:

O = F₁(P₁, S₁) + F₂(P₂, S₂)while a serial configuration results in the following expression:

O = F₂(F₁, (P₁, S₁), S₂)

It is also possible to arrange arbitrarily many displays in a network toform compound displays by taking the union of the component displayparameters and source illuminations as the inputs to the compounddisplay. An example for a parallel configuration is shown in FIG. 10.

Compound displays can consequently be represented as specific types ofabstract displays, which can in turn be arranged into networks and/orgrouped to form higher level compound displays. Provided the componentdisplay image formation models, Fi, are known a mathematical imageformation model of the overall display system can be expressed viacombinations of the serial and parallel formulas. Such an imageformation model may be applied to optimize the operation of a displaysystem.

Display Parameter Optimization

One benefit of this representation is that once the overall imageformation model for the display system is defined, optimal parametersfor individual displays can be obtained via numerical optimization. Suchoptimizations can incorporate multiple, sometimes conflicting, goals inorder to balance desirable properties such as artifact mitigation,maximization of component display lifespans, total system efficiency,power consumption, and output image fidelity among many other options.

Considering a display system as an abstract (possibly compound) displaythat takes parameters, P, and source illumination, S, to produce anoutput image can allow the parameters to be jointly optimized. Such asystem is depicted in FIG. 11, in which display parameter optimizationis performed to determine the parameters, P, and illumination, S,required to reproduce an input target image, T, for an abstract(possibly compound) display. The simulated (or measured) output of thisdisplay is then fed back through the system to several modules: an imagefidelity model, a system constraint model and a quality heuristicsmodel.

Although not explicitly labeled for diagram clarity, the models used bythe system implicitly have access to target image, source illuminationand current parameter selection. A camera located to acquire imagesshowing the output of the display may also be incorporated into thefeedback loop. In some embodiments, optimization is performed using acost function that includes differences between images acquired by thecamera and the desired output of the display system (a target image).

Each of the models attempts to correct for deviations of the outputimage or parameter selection from desirable properties. One common modelis image fidelity: it is desirable that the image produced by the systemclosely approximate the target image, T, or a modified version of thetarget image, perhaps one where perceptual factors are taken intoaccount. Errors between the output image and target image are used bythe model to compute parameter adjustments. Optimization may proceeduntil either convergence of the parameters is achieved or a time budgetis exhausted.

The system constraints model ensures that the parameter selection resultin physically realizable (and desirable configurations). Such criteriacan include requiring that source illumination profiles are within theavailable power or that parameters for modulators vary between opaqueand transmissive, i.e. do not produce light. Desirable configurationsmay include choosing parameters that have spatial or temporal coherence,that are within a certain range (see e.g. the LCoS linearity discussionearlier), or parameters that minimize power usage and/or maximizecomponent lifetime.

Image quality heuristics may be used to compensate for behaviors thatare not easily modeled or which are costly to model for the imageformation models. Image quality heuristics may include moiré,diffraction, temporal behavior and color fringing, among otherartifacts. The heuristics models are intended to help compensate forthese using empirical image-quality criteria. Image quality heuristicscan also be provided to adjust parameters to optimize for properties ofhuman perception, such as veiling luminance, adaptation levels, meanpicture levels, metamerism and variations in sensitivity to chroma/lumaerrors. Sensitivity to these properties can be exploited in contentgeneration.

FIG. 12 shows HDR+LDR projector systems depicted in the above-describedabstract display framework.

The LDR and HDR projectors may themselves be compound displays. Anexample embodiment having desirable properties for commercialapplications has a relatively high power LDR projector that can achievea full-screen white suitable for typical average picture levels combinedwith a lower-power HDR projector that can achieve much higher peakbrightness but does not have the power to do so over the entire screen.Such a system can be vastly more efficient and less costly than buildinga single projector capable of increased full-screen white values due todistributions of luminance in typical images. In such an embodiment, itis desirable to provide a control which permits global dimming of theLDR projector. Some example ways to provide such global dimming use aniris, a controllable shutter, and/or a variable output light source. Theiris is a very simple display that modulates the intensity of the LDRprojector, which could be replaced, in principle by a source, S1, forthe LDR projector that can be dynamically modulated.

The display parameter optimization searches for LDR parameters P1,Iris/drive level parameters P2 and HDR parameters P3 causing the outputimage O to best match the target image T. The system of FIG. 12 thentakes the place of the abstract display in the previous figure, withparameters P={ P1, P2, P3} and S={ S1, S3}. The output image as modeledby the image formation models is then:

O = F₂(P₂, F₁(P₁, S₁)) + F₃(P₃, S₃) = F(P, S)

Improved display parameters can be obtained via optimization. Theoptimization may comprise minimizing the sum of cost functionsrepresenting the image fidelity, image quality and system constraints,for example as follows:

$P = {{{\arg\;\min} \propto {{C( {T - {F( {P,S} )}} )} + {\sum\limits_{i\; \in Q}{\beta_{i}{Q_{i}( {P,S} )}\mspace{14mu}{subject}\mspace{14mu}{to}\mspace{14mu}{K_{j}( {P,S} )}}}}} = {0{\forall j}}}$

Here the image fidelity model is the function, C, which weights errorsbetween the image produced by the system, F(P,S), to produce a scalarindicating how preferable the current set of parameters are. Commonexamples for C are the mean squared error (MSE) or the mean absoluteerror (MAE).

The functions Q_(i) represent image quality heuristics/models which alsoproduce scalar values indicating how preferable the current parametersare in terms of unmodeled artifacts, e.g. moiré, color fringing, ordiffractions artifacts. The constants α and β_(i) control the relativeimportance given to the various terms (which may be contradictory),providing a way for the content generation to favour one objective overanother.

The constraints K_(i) impose conditions on the parameters, for instancethat modulators in projectors must operate in the range between fullytransmissive and fully opaque. They are expressed here as set-valuedconstraints that are either satisfied (K_(j)(P,S)=0) or unsatisfied,however existing optimization techniques can relax these conditions toallow minor constraint violations.

Although not explicitly listed, the constraint functions, K, and imagequality models, Q, may also have a dependence on the output image,O=F(P,S).

It is now possible to express several different schemes for partitioningimage content between the LDR and HDR projectors. Several differentexamples are presented here:

Smooth Blends Between HDR and LDR Projector

Although the HDR projector is necessary for high luminance regions, itmay be desirable, from an image quality perspective, to also make use ofthe HDR projector in regions below the full-screen white level of theLDR projector. This requires portioning content between the twoprojectors.

One straightforward way of approaching this is to blur or diffuse themask used by the HDR projector, for example by blurring a dilated binarymask of pixels above the LDR projector full-screen white. A moresophisticated approach could compute approximations of the veilingluminance at each pixel in order to adjust blending parametersdynamically.

There are numerous other options for how to partition content betweenthe component projectors. Examples of these options are discussed below:

-   -   1) Targeting luminance distributions in which there is a        preferred ratio between the total LDR and HDR projector        contributions (e.g. 95% and 5% respectively), for medium        brightness scenes with high black-levels and highlights.    -   2) Targeting luminance distributions that favour use of the HDR        projector while minimizing use of the LDR projector via dimmable        sources or external irises. Such objectives can potentially        reduce energy use and cooling requirements while also improving        black-levels for dark scenes with bright highlights.    -   3) Targeting temporally consistent luminance distributions for        one or both projectors in order to minimize temporal artifacts.    -   4) Reaching the absolutely widest dynamic range, highest peak        luminance, or deepest black level of the combined display system        in order to maximize perceived image quality.

With any of these approaches, the blending factors may be dynamicallyadjusted spatially within a scene to achieve desired local behaviour.For instance, low luminance content adjacent to high-luminance regionsmay be obscured by veiling luminance of highlights. In this case,neither of the LDR and HDR projectors need to display content for thoseregions. Alternatively, some scenes may have large bright regions andlarge dim regions. The adjustments discussed above can then be made,taking into account the scattering behavior of the projectors.

Extending Color Gamut

If the primary colours used in the HDR and LDR projectors differ,perhaps by design, it may be possible to extend the color gamut of thecombined system. This can be achieved by mapping the target image to theappropriate color-space and determining what mixture of the twoavailable sets of primaries best represents the target color, forinstance choosing as broad a set of primaries as possible to improvemetamerism. The process here is similar in principle to that used inextending the dynamic luminance range, as has been discussed throughoutthis document.

Super-Resolution

If the HDR and LDR projectors are deregistered, it may be possible toincrease the apparent resolution of the combined system to decreasealiasing near edges. This can be achieved by optimizing for a highresolution target image, which will cause the projector contributionsbetween HDR and LDR to automatically adjust in order to best approximatethe high spatial frequency features.

Scatter Compensation & Feedback of Ambient Conditions

Scatter from the viewing environment can lead to dark image regions withelevated levels. Incorporating a heuristic scattering model for eitherthe target or output image allows this to be taken into account in orderto compensate for this effect. In this case the image formation model Fcould be represented as follows:

F(P, S) = F^(′)(P, S) + R(P, S)

Here R is a function modeling scatter from the viewing environment andF′ is the image formation model for the system in a non-scatteringviewing environment. Parameters for the displays optimized using thisimage formation model automatically attempt to compensate for theresulting scatter.

A similar approach can use actual measurements of scattered light inplace of the function R in order to dynamically compensate for lightscattering from the viewing environment.

The method illustrated in FIG. 13 details one approach to determiningwhat image will be shown by what projector, and how they are computed.

The decision boxes depicted in FIG. 13 may incorporate a small amount oftemporal hysteresis such that the LDR and HDR projectors will not bounceback and forth about a threshold from image to image.

The “Tone Map Image” operation examines the luminance levels (ifavailable) in the incoming image and maps them to the capabilities ofthe combined LDR and HDR projector. This operation also takes in accountthe ambient light level when mapping the darker areas of the image, andthe maximum overall luminance the observer would be comfortable with.

The “Adjust Black Level” operation will increase the black level of themapped image in cases where the observer will not be able to perceivethe lower black level. An example of this would be black text in a whitefield where veiling luminance would not allow an observer to distinguisha very low black level from a slightly elevated one. To achieve this, aforward model of the projectors may be used (to predict halo frombrightness).

If an image still has a low black level after the above operations, aniris size (the amount of light attenuated by the iris or by dimming alight source) may be calculated to compensate for the elevated nativeblack level of the LDR projector. Shrinking the iris will also lower thepeak brightness available from the LDR projector. The reduced peakbrightness may be computed as well.

If the LDR projector with its diminished iris size will not supplysufficient light to the image, the HDR projector may be used to generatethe entire image. Note that as explained in the iris section above, itmay be desired to never completely block all light from the LDRprojector.

In the case where black levels are not low and the image containshighlights that cannot be shown using just the LDR projector due toinsufficient brightness capabilities, a separate image for the LDR andthe HDR projector may be computed. Since two images are being combinedon screen in this case, care should be taken to “blend” them such thatedge artifacts are not created when adjacent pixels are delivered fromdifferent projectors. The following approaches may be taken, eitherindividually or in combination:

-   -   threshold banding (always summing pixels)    -   using different gamma curves for each projector    -   spatial variation (slight blur of one projector)    -   temporal dithering

An example of threshold banding would be in the small pixel areassurrounding a bright feature. Here both projectors would contributelight and sum together to create the pixels. The size of this area canbe calculated from the veiling luminance effect or simply a fixed numberof pixels when there is a fairly soft transition between the highlightand the adjacent features (bright spot on a gradient).

Using a Brightness Booster for Multiple Stage Projection

FIG. 14 schematically shows a projection system with two imagingelements in which an auxiliary booster light source is used whenrequired to reproduce certain high brightness and/or low contrastimages.

High dynamic range projectors use two or more imaging stages to lowerblack levels when generating images. Each one of these image stages hasa loss associated with it so when creating very bright images there isfar more light loss in a multi stage projector as compared with a singlestage projector. Light can be added when required before the finalimaging stage to boost the efficiency of the system when low blacklevels are not required.

Image forming elements used in the light path of projection systems arenon-ideal in nature. When forming an image they allow light to leakthrough in dark areas and absorb some light in bright areas at theexpense of overall contrast. To address this, projector manufacturershave made systems with multiple imaging elements to decrease the amountof light leaking through the system in dark areas. This in turn hasrequired a much brighter light source to compensate for the transmissionlosses through two (or more) imaging elements in bright areas. Theseprojectors show dramatically lower operational efficiency when showingbright images as compared with single stage projectors.

A projection system according to the example embodiment in claim 14examines the nature of the image being projected and in the case of alow contrast high brightness image will add a calculated amount ofuniform light before the final imaging stage. The added light will thenonly have to travel through a single imaging stage and thus incur farlower transmission losses. Thus, the operational efficiency of thesystem when producing bright images will be substantially increased.When producing images that require far less light and higher contrast,little or no light will be added before the last imaging elements topreserve the low black levels expected of a multiple stage system.

It is not mandatory that boost light delivered to the second imagingstage be uniform or even. In some embodiments the booster light isnon-uniform. An example application of this is in the case where a firstimaging stage provides a light output that includes undesired lightpatches or other artifacts. For example where the first stage is a lightsteering stage the first stage may provide static artifacts that are notsteerable (for example a global roll-off of intensity towards the edges,or visible patches and stripes from different laser diodes that for onereason or another are not corrected for). In such cases the boosterlight may be structured in such a way that the sum of the booster lightand the artifacts is uniform or near uniform illumination. This may bedone by providing a non-uniform pattern of booster light inverse to thepattern of artifacts from the first stage.

FIG. 14 shows a “main light source” and a “boost light source”. Thelight output of both light sources may be controlled in an independentfashion. The “main light source” is expected to illuminate the firstimaging element in an even, or otherwise defined manner. The “boostlight source” is expected to illuminate the last imaging element.

The purpose of the first imaging element is to block light or steerlight away from darker parts of the image such that the last imagingelement will not have to block much light from darker parts of the imagebeing projected, leading to a high contrast image when desired. Thefirst imaging element may, for example, modulate the phase and/orintensity of light from the main light source.

The “last imaging element” can be paired such that the boost lightsource has its own independent light path to the screen. This may bedesirable in a very high power system when a single final stage imagingelement may not be able to handle the thermal stress or intensityassociated with both light paths being summed on its surface.

In a color projector the methods can be implemented separately for eachcolor primary in the system or operated in a color field sequentialmanner on one or more example implementations.

FIG. 15 is a flow chart illustrating an intensity control method for thelight sources in such a projection system. Such a method may beimplemented in a controller for a display. In an alternative embodimentthe method is implemented in an image processing system that providesoutput image data accompanied by control signals for light sources.

An algorithm is executed to govern the relative intensity settings ofthe two light sources. The boost light will be active when displayinglow contrast imagery or when veiling luminance in the observer's eye orother optical scatter in the system or environment masks surroundingdark areas such that elevating the intensity of those dark areas doesnot result in noticeable degradation of the displayed image.

Image statistics, for example a histogram of the luminance distributionwithin an image, or other methods may be employed to determine theoverall contrast requirements of the image. The boost light source maybe used whenever possible as it is a more efficient light path than fromthe main light source and may always be used to provide brightness up tothe darkest level present in an image.

The main light source may be dimmed to compensate for light being addedto the image by the boost light source.

FIG. 16 illustrates example images with different characteristics suchas peak luminance, mean luminance and black level as well as sensibleintensity levels for a auxiliary (boost) light source.

Cases A and H show an image that is uniformly white at full intensity.In cases B, C, D, I, and J the boost light can drive higher than thelowest level due to veiling luminance effects. Cases P and Q are alsoaffected by veiling luminance and allow some light to come from theboost light. In cases K, L, M, N, and O the boost light drives to thelowest brightness level present in the image. For example, the boostlight may be provided at a level determined by multiplying the lowestluminance level in the image by a factor. The factor may be based on thecontrast capability of the second modulator. For example if the lowestluminance level in a particular image is L_(min)=1 cd/m², and thecontrast of the second modulator C2=2000:1, then the booster light maybe provided with a luminance sufficient to achieve 2000 cd/m² with afully open modulator C2 while allowing the light level to be reduced to1 cd/m² by setting the second modulator to its least light-transmittingstate.

In some embodiments, if a dark patch exceeds a threshold size such thatit will not be masked by a veiling luminance effect, the boost lightwill be completely turned off and the non-black area of the screen willbe illuminated through two image forming elements in series—drasticallyreducing the amount of light leaking through into the dark areas. Inexample cases E, F, G, R, S, T, and U there is enough dark content thatthe boost light is powered off to preserve the black levels.

It is not mandatory that the boost light and the main light source aredistinct from one another. In some embodiments an optical system isprovided that can direct some or all light from a main light sourcedirectly onto the last imaging element bypassing the first imagingelement. For example, a variable beam splitter may be applied to divertsome light from a main light source onto the last imaging element. Someembodiments have both a separate boost light source and a provision fordiverting light from the main light source onto the last imagingelement.

In some embodiments an optical element or elements are provided tocombine light from the boost light source with light that has beenmodulated by the first imaging element and to direct the combined lightonto the last imaging element. The optical element or elements comprisesa prism in some embodiments.

In some embodiments the boost light source comprises a plurality oflight sources such as a plurality of light-emitting diodes (LEDs). Inone example embodiment the boost light source is arranged around anouter perimeter of the first imaging element. For example, the boostlight source may comprise a ring of LEDs. Suitable reflectors,diffusers, spaces and/or other optical elements may be provided to causelight from the boost light source to be evenly distributed on the lastimaging element.

FIGS. 2A to 6A show example images in five cases with differentcharacteristics which are discussed above. The following explains by wayof example how an auxiliary (booster) light source may be controlled foreach of these 5 cases. In an example embodiment, the projector systemused in the following examples may include a high efficient projectorwith steerable light source (main light source and first imagingelement), a secondary imager and a booster stage that illuminates onlythe secondary imager. The secondary imager may, for example, comprise areflective or transmissive spatial light modulator such as a LCD panel,LCOS, DMD, reflective LCD, or the like.

Case 1: Bright Low Dynamic Range Image, Elevated Black Levels

The boost stage is used to illuminate most of the image. The first,steering and high contrast stage is used to add minimal highlights tothe image. Little steering is required.

Case 2: Dim Low Dynamic Range Image, High Blacks

The boost stage is used to illuminate the entire image. The steeringstage is not used.

Case 3: Bright High Dynamic Range Image, High Blacks

The boost stage is full on. The steering stage is also full on providingmaximum steering.

Case 4: Bright High Dynamic Range Image, Low Blacks

The boost stage is off. The image is created using the steering stageonly.

Case 5: Dim Low Dynamic Range Image, Low Blacks

The boost stage in on, but at reduced intensity to preserve some of theblack level in the image. The steering stage is off as no highlights areneeded.

Technology as described herein may be applied, without limitation, todisplays of the types described in U.S. patent application No.61/893,270 filed Oct. 20, 2013 which is hereby incorporated herein byreference for all purposes.

Using a Combination of Projectors to Show Stereoscopic Content

Systems of combined projectors or light sources, as described herein,lend themselves to applications that require the efficient or low costor high brightness reproduction of 3D (stereoscopic) content.

Stereoscopic image pairs comprise an image intended for viewing with theright eye and an image intended for viewing with the left eye. Thedisparity of the images creates a depth effect. No disparity will renderimages perceived to be in the plane of the projection screen. Adisparity between left and right eye images will render objects to beperceived away from the projection screen plane, either closer to theviewer (audience) or, if inverted further away (perceived to be behindthe screen plane).

One characteristic of cinematic and other stereoscopic image content isthat a pleasant viewing experience is more likely to be achieved if thedisparity between left and right eye views is not too great (forexample, depicted objects are not perceived as being too close to theviewer). The differences between the left and right eye views instereoscopic image pairs are therefore typically kept small. Even inimage pairs with depicted content that is perceived as being very closeto the viewer (or very far away), many image areas in the left and righteye views will typically be the same because in almost all cases onlysome objects will be rendered as being close or far relative to theviewer.

Many, if not all, practical stereoscopic projection systems requirefiltering of light that is reflected off the projections screen beforethe light enters each eye of an observer. This filtering results indifferent images being delivered to viewers' left and right eyes.Filtering is often provided using eyeglasses which provide differentfilters for the left and right eyes. Common techniques use color filters(notch filters for some or all of the color primaries for the left andthe right eye), circular or linear polarization filters, temporalshutters or temporal polarization switches.

Projection systems are set up to produce different images for the leftand right eyes which have different corresponding (to the filter atright and left eye) light properties, for example narrow band primariesdifferent for left and right eye view, or clockwise andcounter-clockwise circularly polarized light, or light with orthogonallinear polarization states, or temporal light fields matching thetemporal shutter at the eye or the polarization of the polarizationswitch.

All of these filtering techniques have in common that a large amount oflight is lost between the light source of the projector and theobservers' eye compared to similar non-stereoscopic projection systems.Stereoscopic projection systems are also more complex and thus morecostly than non-stereoscopic projection systems. Another problem is thatit is not always possible or easy to upgrade an existingnon-stereoscopic projector to operate as a stereoscopic projector.

In a system as described herein, it is possible to use one projector ina non-stereoscopic mode with a light source that is compatible with boththe left and the right eye filters (for example a broadband light sourcein the case of a system based on color notch filters, or a randomlypolarized system in the case of either the circular or linearlypolarized filter system or a permanently ON light source in case of anytemporal shutter filtering system). The non-stereoscopic projector willcreate those parts of an image that are common to both the left and theright eye view.

A second projector (one or more projectors) may then be used to displaythe parts of the images that differ between the left and right eyeviews. The second projector projects light having the propertiesrequired for the left and the right eye filters (wavelength, orpolarization, or temporal image fields).

There are several benefits in using such a system: compared to thesystem described herein, the additional cost to enable stereoscopicprojection is minimal, because most of the components are alreadyincluded in the architecture.

The power requirements for the second projector can be lower as theimage regions with disparity between left and right are typically notlarge relative to all pixels of the image. Light steering may be used tosteer light to the display areas corresponding to depicted objectsperceived as being out of the plane of the display screen.

Creating good separation (=contrast) between the left and the right eyeis not easy or costly. Less than perfect separation will result in somelight intended for the right eye entering into the left eye. This effectis known as ghosting and reduces image quality and causes headaches.Since the second projector power requirements are lower than the mainprojector and the cost to make such a second projector is lower, morecare can be taken to ensure that left and right eye views are trulyseparated.

A low power secondary projector can cost effectively be added to upgradeand enable an existing non-stereoscopic projection system to displaystereoscopic images.

Power Output Relationship Between LDR/HDR Projectors

With projector systems as described herein it should be possible tocombine an LDR projector with for example 5× the power of the HDRprojector. Since HDR projectors are far more expensive than LDRprojectors this will allow for a more economical setup.

Non-Limiting Enumerated Example Embodiments

The following are non-limiting enumerated example embodiments.

-   -   1. A method for displaying an image defined by image data, the        method comprising:        -   generating first modulated light by modulating light from a            first light source using a first imaging element;        -   providing boost light;        -   combining the boost light and the first modulated light; and        -   further modulating the combined light using a second imaging            element.    -   2. A method according to aspect 1 wherein combining the boost        light and the first modulated light comprises illuminating a        surface of the second imaging element with both the boost light        and the first modulated light.    -   3. A method according to aspect 1 or 2 wherein combining the        boost light and the first modulated light comprises directing        the boost light and the first modulated light into a prism.    -   4. A method according to aspect 2 wherein the boost light evenly        illuminates the surface of the second imaging element.    -   5. A method according to aspect 2 wherein the boost light is        arranged to provide structured illumination to the surface of        the second imaging element according to a desired luminance        profile.    -   6. A method according to aspect 5 wherein the structured        illumination has higher luminance on some parts of the surface        of the second imaging element than it does in other parts of the        surface of the second imaging element and the luminance of the        highest luminance part of the structured illumination is at        least twice a luminance of lowest luminance parts of the        structured illumination.    -   7. A method according to any one of aspects 1 to 6 wherein        operating the boost light source comprises controlling an output        of light by the boost light source.    -   8. A method according to aspect 7 wherein controlling an output        of light by the boost light source is based at least in part on        a contrast of the image.    -   9. A method according to aspect 8 comprising determining the        contrast of the image by processing an image histogram for the        image.    -   10. A method according to any one of aspects 1 to 9 comprising        dimming the first light source in combination with operating the        boost light source.    -   11. A method according to any one of aspects 1 to 9 comprising        processing the image data to identify any dark patches that        exceed a threshold size and, in response to identifying the dark        patches that exceed the threshold size, turning off the boost        light source.    -   12. A method according to any one of aspects 1 to 11 wherein        generating the boost light comprises operating a boost light        source separate from the first light source.    -   13. A method according to any one of aspects 1 to 11 wherein        generating the boost light comprises directing light from the        first light source onto the second imaging element.    -   14. A method according to aspect 13 wherein directing light from        the first light source onto the second imaging element comprises        controlling a variable beam splitter.    -   15. A method according to aspect 13 wherein directing light from        the first light source onto the second imaging element comprises        delivering the light by way of an switch having one input port        arranged to receive light from the first light source and two or        more output ports, one of the output ports arranged to deliver        the light to the second imaging element.    -   16. A method according to aspect 13 or 15 comprising adjusting        the amount of boost light delivered to the second imaging        element by time division multiplexing.    -   17. A method according to any one of aspects 1 to 16 comprising        processing the image data to determine a lowest luminance level        present in the image and operating the boost light source at a        level corresponding to the lowest luminance level in the image.    -   18. A method according to any one of aspects 1 to 16 comprising        processing the image data to simulate veiling luminance,        determining a lowest perceptible luminance level present in the        image and operating the boost light source at a level        corresponding to the lowest perceptible luminance level.    -   19. A method according to any one of aspects 1 to 17 wherein the        second imaging element comprises a spatial light modulator.    -   20. A method according to any one of aspects 1 to 17 wherein the        second imaging element comprises a LCD panel, LCOS, reflective        LCD panel, or DMD.    -   21. A method for generating signals for controlling a projector        to display images according to image data, the projector        comprising a first imaging element configured to provide        modulated light to a second imaging element for further        modulation by the second imaging element and a boost light        configured to deliver additional illumination for modulation by        the second imaging element, the method comprising: simulating        veiling luminance to determine a lowest perceivable luminance        level in the image and generating a signal to set the boost        light at a level corresponding to the lowest perceptible        luminance level.    -   22. A method according to aspect 21 comprising performing the        step of simulating veiling luminance in response to the image        data satisfying a condition.    -   23. A method according to aspect 22 comprising processing the        image data to determine a contrast of the image wherein the        condition comprises determining that the contrast is lower than        a threshold value.    -   24. A method according to aspect 22 or 23 wherein the method        comprises detecting any dark features in the image and the        condition comprises determining that all of the dark features        are smaller than a threshold size.    -   25. A method according to aspect 24 comprising, if any of the        dark features are larger than the threshold size, generating a        signal to set the boost light source to be off.    -   26. A method according to any one of aspects 21 to 23 comprising        processing the image data to detect dark features in the image        data, the method comprising, if any of the dark features are        larger than the threshold size, generating a signal to set the        boost light source to be off.    -   27. A method according to any one of aspects 21 to 26 comprising        processing the image data to determine an amount of the image        that is dark and, if the image is predominantly dark, generating        a signal to set the boost light source to be off.    -   28. A method according to any one of aspects 21 to 27 comprising        generating the signal to set the boost light at a level        corresponding to the lowest perceptible luminance level in        combination with generating a signal to reduce a level of a main        light source illuminating the first imaging element.    -   29. A method according to any one of aspects 21 to 27 performed        by a controller in the projector.    -   30. A method according to any one of aspects 21 to 27 performed        by an image processing system configured to provide output image        data accompanied by control signals for the boost light.    -   31. A method according to any one of aspects 21 to 30 wherein        the boost light uniformly illuminates the second modulator.    -   32. A method according to any one of aspects 21 to 30 wherein        the boost light non-uniformly illuminates the second modulator.    -   33. A light projector comprising:        -   a first imaging element configured to provide modulated            light to a second imaging element for further modulation by            the second imaging element and a boost light configured to            deliver to the second imaging element illumination for            modulation by the second imaging element.    -   34. A light projector according to aspect 33 wherein the first        imaging element is configured to modulate one or both of the        phase and amplitude of light from a main light source.    -   35. A light projector according to aspect 33 or 34 wherein the        boost light is separate from the main light source.    -   36. A light projector according to aspect 35 wherein the boost        light comprises a plurality of light sources.    -   37. A light projector according to aspect 35 wherein the        plurality of light sources of the boost light comprises a        plurality of light emitting diodes (LEDs).    -   38. A light projector according to aspect 35 wherein the        plurality of light sources of the boost light comprises a        plurality of laser diodes.    -   39. A light projector according to any one of aspects 36 to 38        wherein the plurality of light sources of the boost light are        arranged around an outer perimeter of the first imaging element.    -   40. A light projector according to any one of aspects 36 to 39        wherein the light sources of the boost light are arranged in a        ring.    -   41. A light projector according to any one of aspects 36 to 40        wherein the light sources of the boost light are individually        controllable to yield a desired pattern of boost light on the        second imaging element.    -   42. A light projector according to any one of aspects 33 to 34        wherein the boost light comprises an optical system configured        to direct light from the main light source directly onto the        second imaging element.    -   43. A light projector according to aspect 42 wherein the optical        system comprises a variable beam splitter.    -   44. A light projector according to any one of aspects 33 to 43        comprising a controller configured to process the image data and        to output control signals for the first and second imaging        elements and the boost light.    -   45. A light projector according to aspect 44 wherein the        controller is configured to simulate veiling luminance to        determine a lowest perceivable luminance level in the image and        set the boost light at a level corresponding to the lowest        perceptible luminance level.    -   46. A light projector according to aspect 45 wherein the        controller is configured to perform the step of simulating        veiling luminance in response to the image data satisfying a        condition.    -   47. A light projector according to aspect 46 wherein the        controller is configured to process the image data to determine        a contrast of the image wherein the condition comprises        determining that the contrast is lower than a threshold value.    -   48. A light projector according to aspect 46 or 47 wherein the        controller is configured to detect any dark features in the        image and the condition comprises determining that all of the        dark features are smaller than a threshold size.    -   49. A light projector according to aspect 48 wherein the        controller is configured to, if any of the dark features are        larger than the threshold size, set the boost light source to be        off.    -   50. A light projector according to any one of aspects 47 to 49        wherein the controller is configured to process the image data        to detect dark features in the image data, and, if any of the        dark features are larger than the threshold size, set the boost        light source to be off.    -   51. A light projector according to any one of aspects 47 to 50        wherein the controller is configured to process the image data        to determine an amount of the image that is dark and, if the        image is predominantly dark, set the boost light source to be        off.    -   52. A light projector according to any one of aspects 47 to 51        wherein the controller is configured to set the boost light at a        level corresponding to the lowest perceptible luminance level in        combination with reducing a level of illumination of the first        imaging element.    -   53. A light projector according to any one of aspects 33 to 52        wherein the second imaging element comprises a spatial light        modulator.    -   54. A light projector according to any one of aspects 33 to 52        wherein the second imaging element comprises a LCD panel, LCOS,        reflective LCD panel, or DMD.    -   55. A light projection method comprising controlling a plurality        of imaging stages arranged in series to produce modulated light        and selectively adding light before a final one of the imaging        stages when low black levels are not required.    -   56. A light projection method according to aspect 55 comprising        processing image data to determine a contrast of an image        represented by the image data, the method comprising adding the        light when the contrast is below a threshold value.    -   57. A light projection method according to aspect 56 comprising        determining the contrast by processing an image histogram.    -   58. A light projection method according to any one of aspects 55        to 57 comprising uniformly distributing the added light at the        final one of the imaging stages.    -   59. A light projection method according to any one of aspects 55        to 58 wherein controlling the plurality of imaging stages        comprises controlling the imaging stages to modulate one or more        of the phase and amplitude of light incident on the imaging        stage.    -   60. A light projection method according to any one of aspects 55        to 59 comprising varying the amount of added light based on data        defining an image to be projected.    -   61. A light projection method according to any one of aspects 55        to 57 comprising non-uniformly distributing the added light at        the final one of the imaging stages.    -   62. A light projection method according to aspect 61 comprising        structuring the added light such that the added light summed        with artifacts from earlier imaging stages yield uniform        illumination of the final one of the imaging stages.    -   63. A light projector comprising:        -   a first imaging stage arranged to modulate light from a main            light source;        -   a second imaging stage arranged to further modulate light            modulated by the first imaging element; and        -   a boost light arranged to add light after the first imaging            stage and before the second imaging stage such that the            added light is modulated by the second imaging stage; and        -   a controller operative to process image data and to operate            the boost light when low black levels are not required.    -   64. A light projector according to aspect 63 wherein the        controller is configured to process the image data to determine        a contrast of an image represented by the image data and to        operate the booster light to add light when the contrast is        below a threshold value.    -   65. A light projector according to aspect 64 wherein the        controller is configured to determine the contrast by processing        an image histogram.    -   66. A light projector according to any one of aspects 63 to 65        wherein the booster light is arranged to evenly illuminate the        second imaging stage.    -   67. A light projector according to any one of aspects 63 to 65        wherein the first imaging stage is controllable to modulate one        or more of the phase and amplitude of light incident on the        first imaging stage.    -   68. A light projector according to any one of aspects 63 to 67        wherein the controller is configured to vary the amount of light        added by the booster light based on the image data.    -   69. A method for projecting a light pattern defined by image        data, the method comprising:        -   generating first modulated light by modulating light from a            first light source using a first imaging element;        -   providing boost light;        -   further modulating the first modulated light and modulating            the boost light; and        -   combining the modulated boost light and the further            modulated first modulated light.    -   70. A method according to aspect 69 wherein combining the        modulated boost light and the further modulated first modulated        light comprises projecting the modulated boost light and the        further modulated first modulated light onto a surface.    -   71. A method according to aspect 69 or 70 wherein the modulated        boost light has a higher black level than the further modulated        first modulated light.    -   72. A method according to any one of aspects 69 to 71 wherein        the modulated boost light has a higher peak luminance than the        further modulated first modulated light.    -   73. A method according to any one of aspects 69 to 72 wherein        the modulated boost light has a lower dynamic range than the        further modulated first modulated light.    -   74. A method according to any one of aspects 69 to 73 wherein        further modulating the first modulated light and modulating the        boost light are both performed with a second imaging element.    -   75. A method according to any one of aspects 69 to 74 wherein        further modulating the first modulated light and modulating the        boost light both apply the same modulation.    -   76. A method according to aspect 75 comprising evenly        illuminating a surface of the second imaging element with the        boost light.    -   77. A method according to any one of aspects 69 to 76 wherein        providing the boost light comprises controlling an output of        light by a boost light source.    -   78. A method according to aspect 77 wherein controlling an        output of light by the boost light source is based at least in        part on a contrast of the image data.    -   79. A method according to aspect 78 comprising determining the        contrast of the image data by processing an image histogram for        the image data.    -   80. A method according to any one of aspects 69 to 79 comprising        dimming the first modulated light in combination with providing        the boost light.    -   81. A method according to any one of aspects 69 to 80 comprising        processing the image data to identify any dark patches that        exceed a threshold size and, in response to identifying the dark        patches that exceed the threshold size, turning off the boost        light.    -   82. A method according to aspect 75 comprising non-evenly        illuminating a surface of the second imaging element with the        boost light.    -   83. A method according to any one of aspects 69 to 82 wherein        providing the boost light comprises operating a boost light        source separate from the first light source.    -   84. A method according to any one of aspects 69 to 82 wherein        providing the boost light comprises directing light from the        first light source onto a second light modulator.    -   85. A method according to aspect 84 wherein directing light from        the first light source onto the second light modulator comprises        controlling a variable beam splitter.    -   86. A method according to any one of aspects 69 to 85 comprising        processing the image data to determine a lowest luminance level        present and providing the boost light at a level corresponding        to the lowest luminance level.    -   87. A method according to any one of aspects 69 to 85 comprising        processing the image data to simulate veiling luminance,        determining a lowest perceptible luminance level present in the        image and providing the boost light at a level corresponding to        the lowest perceptible luminance level.    -   88. A projector system comprising a plurality of projectors, the        plurality of projectors comprising at least a first projector        and a second projector arranged such that light projected by the        first and second projectors is combined into a projected image        for viewing wherein the first and second projector have        different imaging characteristics selected from: dynamic range,        black level and peak luminance.    -   89. A projector system according to aspect 88 comprising a        control system connected to receive image data defining image        content to be projected by the projector system and to control        the projector system to project the image content        -   wherein the control system is configured to process the            image data and to generate modified image data for            projection by at least one of the first and second            projectors.    -   90. A projector system according to aspect 89 wherein the        control system is configured to process the image data to        determine dynamic range, black levels and average luminance        level and to generate the modified image data based on the        dynamic range, black levels and maximum luminance level.    -   91. A projector system according to aspect 90 wherein the first        projector has a higher dynamic range, higher peak luminance and        lower black level than the second projector.    -   92. A projector system according to aspect 91 wherein, in the        case where the image data has luminance in higher luminance        areas greater than a maximum luminance of the second projector        the control system controls the luminance threshold to cause the        first projector to project light in at least the higher        luminance areas.    -   93. A projector system according to aspect 92 wherein, in the        case where black levels are above a black level threshold, the        control system is configured to control the second projector to        project as much light of the image as is within the capability        of the second projector.    -   94. A projector system according to any one of aspects 91 to 93        wherein the control system is configured to generate the        modified image data for the first projector by a method        comprising creating a binary mask of pixels having luminances        above the full-screen white value of the second projector.    -   95. A projector system according to aspect 94 wherein the        control system is configured to dilate and blur the binary mask.    -   96. A projector system according to any one of aspects 91 to 95        wherein the control system is configured to generate the        modified image data for the second projector by a method        comprising clipping luminance of pixels in the image data having        luminance values above the full-screen white value of the second        projector.    -   97. A projector system according to any one of aspects 89 to 96        wherein the control system is configured to supply the image        data to the second projector unmodified in the case where the        dynamic range, black levels and average luminance level are        within the capabilities of the second projector.    -   98. A projector system according to any one of aspects 89 to 97        wherein the second projector comprises a controllable iris and        the control system is configured to control the iris to reduce a        black level of the second projector in at least some cases where        the black level of the image data is below a black level of the        second projector.    -   99. A projector system according to any one of aspects 89 to 98        wherein the control system comprises an image formation model        for the projector system and the control system is configured to        obtain values of control parameters for the first and second        projectors by performing an optimization.    -   100. A projector system according to aspect 99 wherein        performing the optimization comprises minimizing a sum of cost        functions.    -   101. A projector system according to aspect 100 wherein the cost        functions include cost functions relating to image fidelity,        image quality and system constraints.    -   102. A projector system according to aspect 101 wherein the cost        function relating to image fidelity comprises a mean squared        error value or a mean absolute error value.    -   103. A projector system according to aspect 101 or 102 wherein        the cost function relating to image quality comprises one or        more heuristics indicating how preferable a current set of        control parameters is in relation to artifacts not modelled by        the image formation model.    -   104. A projector system according to aspect 103 wherein the        heuristics comprise heuristics for one or more of moire, color        fringing and diffraction artifacts.    -   105. A projector system according to any one of aspects 101 to        104 wherein the constraints limit the values of the control        parameters to parameters that are physically realizable.    -   106. A projector system according to any one of aspects 99 to        105 wherein the control system is configured to attempt to        achieve a desired ratio of total light output of the first and        second projectors.    -   107. A projector system according to any one of aspects 99 to        105 wherein the control system is biased to control one of the        first and second projectors to contribute as much light to the        projected image as it is capable of.    -   108. A projector system according to any one of aspects 99 to        107 wherein the image formation model includes a heuristic        scattering model.    -   109. A projector system according to any one of aspects 89 to        108 wherein the first and second projectors have different        primary colors and the controller is configured to balance light        output by the first and second projectors to achieve colours in        the projected image that are outside of a gamut of at least one        of the first and second projectors.    -   110. A projector system according to any one of aspects 89 to        109 wherein the controller is configured to balance light output        by the first and second projectors to achieve an optimized        reproduction of high-spatial frequency features in image content        of the projected image.    -   111. A projector system according to any one of aspects 89 to        110 wherein the control parameters include pixel values for the        first and second projectors.    -   112. A projector system according to any one of aspects 89 to        111 wherein the control parameters include light source values        for the first and second projectors.    -   113. A projector system according to any one of aspects 89 to        112 wherein the control system is configured to take into        account ambient light in an area of the projected image.    -   114. Methods or apparatus according to any one of the above        aspects applied to project light in a vehicle headlight.    -   115. Methods and apparatus according to any one of the above        aspects involving combining light projected from a 2D projector        with light containing a stereoscopic image pair projected by one        or more other projectors wherein left-eye and right-eye images        of the stereoscopic image pair are distinguishable from one        another in one or both of time and distinguishable light        characteristics and the light projected by the 2D projector        comprises light matching both of the left and right-eye images.    -   116. Apparatus having any new and inventive feature, combination        of features, or sub-combination of features as described herein.    -   117. Methods having any new and inventive steps, acts,        combination of steps and/or acts or sub-combination of steps        and/or acts as described herein.        Interpretation of Terms

Unless the context clearly requires otherwise, throughout thedescription and the claims:

-   -   “comprise”, “comprising”, and the like are to be construed in an        inclusive sense, as opposed to an exclusive or exhaustive sense;        that is to say, in the sense of “including, but not limited to”;    -   “connected”, “coupled”, or any variant thereof, means any        connection or coupling, either direct or indirect, between two        or more elements; the coupling or connection between the        elements can be physical, logical, or a combination thereof;    -   “herein”, “above”, “below”, and words of similar import, when        used to describe this specification, shall refer to this        specification as a whole, and not to any particular portions of        this specification;    -   “or”, in reference to a list of two or more items, covers all of        the following interpretations of the word: any of the items in        the list, all of the items in the list, and any combination of        the items in the list;    -   the singular forms “a”, “an”, and “the” also include the meaning        of any appropriate plural forms.

Words that indicate directions such as “vertical”, “transverse”,“horizontal”, “upward”, “downward”, “forward”, “backward”, “inward”,“outward”, “vertical”, “transverse”, “left”, “right”, “front”, “back”,“top”, “bottom”, “below”, “above”, “under”, and the like, used in thisdescription and any accompanying claims (where present), depend on thespecific orientation of the apparatus described and illustrated. Thesubject matter described herein may assume various alternativeorientations. Accordingly, these directional terms are not strictlydefined and should not be interpreted narrowly.

Embodiments of the invention may be implemented using specificallydesigned hardware, configurable hardware, programmable data processorsconfigured by the provision of software (which may optionally comprise“firmware”) capable of executing on the data processors, special purposecomputers or data processors that are specifically programmed,configured, or constructed to perform one or more steps in a method asexplained in detail herein and/or combinations of two or more of these.Examples of specifically designed hardware are: logic circuits,application-specific integrated circuits (“ASICs”), large scaleintegrated circuits (“LSIs”), very large scale integrated circuits(“VLSIs”), and the like. Examples of configurable hardware are: one ormore programmable logic devices such as programmable array logic(“PALs”), programmable logic arrays (“PLAs”), and field programmablegate arrays (“FPGAs”)). Examples of programmable data processors are:microprocessors, digital signal processors (“DSPs”), embeddedprocessors, graphics processors, math co-processors, general purposecomputers, server computers, cloud computers, mainframe computers,computer workstations, and the like. For example, one or more dataprocessors in a control circuit for a device may implement methods asdescribed herein by executing software instructions in a program memoryaccessible to the processors.

While processes or blocks are presented in a given order, alternativeexamples may perform routines having steps, or employ systems havingblocks, in a different order, and some processes or blocks may bedeleted, moved, added, subdivided, combined, and/or modified to providealternative or subcombinations. Each of these processes or blocks may beimplemented in a variety of different ways. Also, while processes orblocks are at times shown as being performed in series, these processesor blocks may instead be performed in parallel, or may be performed atdifferent times.

The invention may also be provided in the form of a program product. Theprogram product may comprise any non-transitory medium which carries aset of computer-readable instructions which, when executed by a dataprocessor, cause the data processor to execute a method of theinvention. Program products according to the invention may be in any ofa wide variety of forms. The program product may comprise, for example,non-transitory media such as magnetic data storage media includingfloppy diskettes, hard disk drives, optical data storage media includingCD ROMs, DVDs, electronic data storage media including ROMs, flash RAM,EPROMs, hardwired or preprogrammed chips (e.g., EEPROM semiconductorchips), nanotechnology memory, or the like. The computer-readablesignals on the program product may optionally be compressed orencrypted.

In some embodiments, the invention may be implemented in software. Forgreater clarity, “software” includes any instructions executed on aprocessor, and may include (but is not limited to) firmware, residentsoftware, microcode, and the like. Both processing hardware and softwaremay be centralized or distributed (or a combination thereof), in wholeor in part, as known to those skilled in the art. For example, softwareand other modules may be accessible via local memory, via a network, viaa browser or other application in a distributed computing context, orvia other means suitable for the purposes described above.

Where a component (e.g. a software module, processor, assembly, display,iris, device, circuit, etc.) is referred to above, unless otherwiseindicated, reference to that component (including a reference to a“means”) should be interpreted as including as equivalents of thatcomponent any component which performs the function of the describedcomponent (i.e., that is functionally equivalent), including componentswhich are not structurally equivalent to the disclosed structure whichperforms the function in the illustrated exemplary embodiments of theinvention.

Specific examples of systems, methods and apparatus have been describedherein for purposes of illustration. These are only examples. Thetechnology provided herein can be applied to systems other than theexample systems described above. Many alterations, modifications,additions, omissions, and permutations are possible within the practiceof this invention. This invention includes variations on describedembodiments that would be apparent to the skilled addressee, includingvariations obtained by: replacing features, elements and/or acts withequivalent features, elements and/or acts; mixing and matching offeatures, elements and/or acts from different embodiments; combiningfeatures, elements and/or acts from embodiments as described herein withfeatures, elements and/or acts of other technology; and/or omittingcombining features, elements and/or acts from described embodiments.

It is therefore intended that the following appended claims and claimshereafter introduced are interpreted to include all such modifications,permutations, additions, omissions, and sub-combinations as mayreasonably be inferred. The scope of the claims should not be limited bythe preferred embodiments set forth in the examples, but should be giventhe broadest interpretation consistent with the description as a whole.

What is claimed is:
 1. An image projection system, comprising: an imagestorage mechanism that stores one or more images to be presented insequence on a projection screen; a high dynamic range projectorincluding a light source that only directly projects light to a firstimaging element that is configured to modulate the phase of light fromthe light source for producing images on a projection screen with anaverage brightness over an entire area of the projection screen andimages with a higher than average peak brightness over less than theentire area of the projection screen; a low dynamic range projectorincluding a light source that only projects light to a second imagingelement that is different than the first imaging element that isconfigured to modulate the intensity of light from the first imagingelement and the intensity of light from the light source in the lowdynamic range projector, wherein the low dynamic range projector has asmaller dynamic range than the high dynamic range projector; and controlhardware that is configured to analyze data for each image to beprojected onto the projection screen on a frame by frame basis toselectively control the low dynamic range projector to supply anapproximately uniform amount of light onto the second imaging elementsuch that some images are projected onto the projection screen usingonly the high dynamic range projector and some images requiring a higherthan average brightness level over the entire area of the projectionscreen than is available from the high dynamic range projector but witha reduced dynamic range are projected on the screen using both the highdynamic range projector and the low dynamic range projector.
 2. Theimage projection system of claim 1, further comprising: a camerapositioned to obtain measurements of the display and to provide themeasurements to the control hardware to adjust the parameters forcreating an image by the high dynamic range projector and by the lowdynamic range projector.
 3. The image projection system of claim 1,further comprising an iris positioned in an optical path between the lowdynamic range projector and the projection screen.
 4. The imageprojection system of claim 1, wherein the light source of the lowdynamic range projector comprises a variable intensity light source. 5.The image projection system of claim 1, wherein the control hardware isconfigured to determine parameters for creating the image to beprojected by the high dynamic range projector and by the low dynamicrange projector based on an image fidelity model comparing the combinedimage with a target image.
 6. The image projection system of claim 1,wherein the control hardware is configured to determine the parametersfor creating the image to be projected by the high dynamic rangeprojector and by the low dynamic range projector based on a systemconstraint model.
 7. The image projection system of claim 1, wherein thecontrol hardware is configured to determine the parameters for creatingthe image to be projected by the high dynamic range projector and by thelow dynamic range projector based on a quality heuristics model.
 8. Theimage projection system of claim 1, wherein the control hardware isconfigured to determine a lowest luminance level present in the image tobe projected and to operate the light source of the high dynamicprojector at a level corresponding to the lowest luminance level in theimage.
 9. The image projection system of claim 1, wherein the controlhardware is configured to simulate veiling luminance, determine a lowestperceptible luminance level present in the image to be projected andoperate the light source of the low dynamic range projector at a levelcorresponding to the lowest perceptible luminance level.